(1). Field of the Invention
This invention relates to a method, apparatus and recording medium for use in identifying pixels in a digital image which are not properly converted owing to dirt or dust (hereinafter called defective pixels) when an original image such as a reversal film or print is converted into the digital image by a reading device such as a scanner. In particular, the invention relates to a technique of identifying defective pixels in a digital image with little effort.
This invention also relates to a method, apparatus and recording medium for use in correcting defective pixels identified in a digital image, and particularly to a technique of preventing loss of the texture of an original image due to the correction of defective pixels.
(2). Description of the Related Art
When an original image such as a reversal film or color print is converted into a digital image by a reading device such as a scanner, dirt or dust may be adhering to the original image or to the reading device. Then, the portion of the original image carrying or corresponding to the dirt or dust is not properly converted into a digital image but becomes unwanted defective pixels. That is, the digital image converted in such a condition includes unwanted defective pixels as well as normal pixels properly converted from the original image. The digital image including such unwanted defective pixels, as it is, has poor quality. Usually, the defective pixels are removed for improved quality.
Such defective pixels, ideally, are removed fully automatically without requiring a manual operation, but such technical level has not been attained to date. Thus, an operation requiring little effort of the operator and a digital image giving no visual incongruity after removal of defective pixels are the two technical points at issue today.
The operation to remove defective pixels includes a plurality of steps, e.g. identifying defective pixels, and correcting the defective pixels identified. Among these steps, what burdens the operator is the step of identifying defective pixels present in a digital image. Varied methods of identifying defective pixels have been in practical use, which are set out hereunder in the order of weight of burden. The following operations are carried out by the operator using a mouse while observing a digital image appearing on a display, for example:
(1) Designate one defective pixel after another.
(2) Mark a free curve around the defective pixels.
(3) Designate two points to form a rectangular or circular area including the defective pixels.
The following method has been proposed as one alleviating the operator's burden, but has not been in practical use yet:
(4) Designate a defective pixel or a position adjacent thereto.
Obviously, among the above methods, the method (4) requires only a single designating operation and therefore imposes the least burden on the operator. The following obstacles exist in implementation of this proposed method.
Assume that, as shown in FIG. 1, a fixed evaluation area ER is set around one defective pixel DP1 designated, and all the pixels in the evaluation area ER are identified as defective pixels. Then, a defective pixel DP2 (not designated) in this area may automatically be identified as a defective pixel, but normal pixels NP in the evaluation area ER also are identified as defective pixels. When the values of the defective pixels (including normal pixels NP and defective pixels DP1 and DP2) identified are corrected such as by interpolation from the values of normal pixels present outside the evaluation area ER, the normal pixels NP requiring no correction are corrected also. The resulting digital image tends to lose the texture of the original image.
It is conceivable to identify the defective pixels in the evaluation area ER based on a certain threshold value, instead of identifying all the pixels in the evaluation area ER as defective pixels as noted above. As shown in FIG. 1, the evaluation area ER may include dark normal pixels NP' representing a shadow or black wall. If defective pixels are discriminated based on a threshold value simply determined, these normal pixels NP' will also be identified as defective pixels. Here again, the texture of the original image tends to be lost.
These drawbacks are obstructive to implementation of the proposed method (4).
As noted hereinbefore, it is an important technical consideration also to realize, after removal of defective pixels, a digital image including no unnatural level difference around the defective pixels, and thus giving no visual incongruity. That is, the texture of an original image should be maintained as much as possible.
The methods of identifying defective pixels include those described above. The following may be cited as typical methods of correcting the defective pixels identified:
(a) Pixel Copying Method: PA1 (b) Method of Correcting Defective Pixel Based on Pixel Values of Surrounding Normal Pixels: PA1 (a) designating a pixel adjacent the defective pixels in the digital image; PA1 (b) setting, as an evaluation area, a group of pixels included in a predetermined area size centering on the pixel designated; PA1 (c) setting, as a defective pixel candidate, a pixel having the darkest value among pixels in a central portion of the evaluation area; PA1 (d) computing line averages of pixels around the defective pixel candidate, among pixels in the evaluation area, the line averages being average pixel values, respectively, of upper lines of pixels arranged horizontally above the defective pixel candidate, lower lines of pixels arranged horizontally below the defective pixel candidate, left lines of pixels arranged vertically and leftward of the defective pixel candidate, and right lines of pixels arranged vertically and rightward of the defective pixel candidate; PA1 (e) diminishing the evaluation area by selecting a line having the lightest line average from each group of the upper lines, the lower lines, the left lines and the right lines; PA1 (f computing a threshold pixel value for determining the defective pixels, from an average pixel value of a rectangular frame defined by the lines selected one each for upper, lower, left and right sides of the rectangular frame; and PA1 (g) comparing values of all pixels in the rectangular frame with the threshold pixel value, and determining pixels having values darker than the threshold value to be the defective pixels. PA1 a storage device for storing the digital image; PA1 a display device for displaying the digital image; PA1 a designating device for designating a pixel adjacent the defective pixels in the digital image displayed on the display device; PA1 an evaluation area setting device for setting, as an evaluation area, a group of pixels included in a predetermined area size centering on the pixel designated; PA1 a defective pixel candidate setting device for setting, as a defective pixel candidate, a pixel having the darkest value among pixels in a central portion of the evaluation area; PA1 an average computing device for computing line averages of pixels around the defective pixel candidate, among pixels in the evaluation area, the line averages being average pixel values, respectively, of upper lines of pixels arranged horizontally above the defective pixel candidate, lower lines of pixels arranged horizontally below the defective pixel candidate, left lines of pixels arranged vertically and leftward of the defective pixel candidate, and right lines of pixels arranged vertically and rightward of the defective pixel candidate; PA1 an area limiting device for diminishing the evaluation area by selecting a line having the lightest line average from each group of the upper lines, the lower lines, the left lines and the right lines; PA1 a threshold computing device for computing a threshold pixel value for determining the defective pixels, from an average pixel value of a rectangular frame defined by the lines selected one each for upper, lower, left and right sides of the rectangular frame; and PA1 a defective pixel discriminating device for comparing values of all pixels in the rectangular frame with the threshold pixel value, and determining pixels having values darker than the threshold value to be the defective pixels. PA1 (a) designating a pixel adjacent the defective pixels in the digital image; PA1 (b) setting, as an evaluation area, a group of pixels included in a predetermined area size centering on the pixel designated; PA1 (c) setting, as a defective pixel candidate, a pixel having the darkest value among pixels in a central portion of the evaluation area; PA1 (d) computing line averages of pixels around the defective pixel candidate, among pixels in the evaluation area, the line averages being average pixel values, respectively, of upper lines of pixels arranged horizontally above the defective pixel candidate, lower lines of pixels arranged horizontally below the defective pixel candidate, left lines of pixels arranged vertically and leftward of the defective pixel candidate, and right lines of pixels arranged vertically and rightward of the defective pixel candidate; PA1 (e) diminishing the evaluation area by selecting a line having the lightest line average from each group of the upper lines, the lower lines, the left lines and the right lines; PA1 (f) computing a threshold pixel value for determining the defective pixels, from an average pixel value of a rectangular frame defined by the lines selected one each for upper, lower, left and right sides of the rectangular frame; and PA1 (g) comparing values of all pixels in the rectangular frame with the threshold pixel value, and determining pixels having values darker than the threshold value to be the defective pixels. PA1 (a) identifying the defective pixels in the digital image; PA1 (b) computing, for each of the defective pixels, the number of defective pixels consecutive in a direction across rows as a consecution number in the direction across rows, and the number of defective pixels consecutive in a direction across columns as a consecution number in the direction across columns; PA1 (c) determining a minimum consecution number among the consecution numbers in the direction across rows and the consecution numbers in the direction across columns, and determining a defective pixel having the minimum consecution number to be a minimum defective pixel; PA1 (d) computing a corrected value for the minimum defective pixel from values of normal pixels opposed to each other across the minimum defective pixel in a direction of the minimum consecution number; PA1 (e) substituting the corrected value for a value of the minimum defective pixel; PA1 (f) regarding the minimum defective pixel as a normal pixel; and PA1 (g) repeating steps (b) through (f) until all of the defective pixels become normal pixels. PA1 a storage device for storing the digital image; PA1 a display device for displaying the digital image; PA1 a defective pixel identifying device for identifying the defective pixels in the digital image displayed on the display device; PA1 a consecution number computing device for computing, for each of the defective pixels, the number of defective pixels consecutive in a direction across rows as a consecution number in the direction across rows, and the number of defective pixels consecutive in a direction across columns as a consecution number in the direction across columns; PA1 a minimum defective pixel computing device for determining a minimum consecution number among the consecution numbers in the direction across rows and the consecution numbers in the direction across column, and determining a defective pixel having the minimum consecution number to be a minimum defective pixel; PA1 a corrected pixel value computing device for computing a corrected value for the minimum defective pixel from values of normal pixels opposed to each other across the minimum defective pixel in a direction of the minimum consecution number; PA1 a pixel correcting device for substituting the corrected value for a value of the minimum defective pixel, the minimum defective pixel being now regarded as a normal pixel; and PA1 a control device for repeatedly controlling the consecution number computing device, the minimum defective pixel computing device, the corrected pixel value computing device and the pixel correcting device until all of the defective pixels become normal pixels. PA1 (a) identifying the defective pixels in the digital image; PA1 (b) computing, for each of the defective pixels, the number of defective pixels consecutive in a direction across rows as a consecution number in the direction across rows, and the number of defective pixels consecutive in a direction across columns as a consecution number in the direction across columns; PA1 (c) determining a minimum consecution number among the consecution numbers in the direction across rows and the consecution numbers in the direction across column, and determining a defective pixel having the minimum consecution number to be a minimum defective pixel; PA1 (d) computing a corrected value for the minimum defective pixel from values of normal pixels opposed to each other across the minimum defective pixel in a direction of the minimum consecution number; PA1 (e) substituting the corrected value for a value of the minimum defective pixel; PA1 (f) regarding the minimum defective pixel as a normal pixel; and PA1 (g) repeating steps (b) through (f) until all of the defective pixels become normal pixels.
For example, the operator visually identifies one defective pixel after another, and copies the value of a normal pixel adjacent each defective pixel for use as the value of the defective pixel.
This method has probably been used most extensively since electronic image processing began to be practiced. However, this method requires a perfect agreement between the values of normal pixels adjacent a defective pixel identified and the tone of the normal pixel copied. Without such agreement, a level difference in tone will occur with the corrected defective pixel, thereby impairing texture. A defective pixel correcting operation based on this method, therefore, imposes a heavy burden on the operator, and requires a high level of skill. This poses a problem that the operator's capabilities are manifestly reflected in image quality. Naturally, a large number of defective pixels will require a long working time.
This method has advantages over the above pixel copying method in that the burden on the operator is alleviated, the operator's skill is immaterial, and a large number of defective pixels may be processed in a short time.
With this method, defective pixels are identified by one of the above methods (1)-(4), a corrected value is derived from the values of normal pixels surrounding the defective pixels, and is substituted for the values of the defective pixels. This method will particularly be described with reference to FIG. 2. A digital image F includes defective pixels DP (marked with "X" signs) as well as normal pixels NP. An average value of those normal pixels (marked with circles) surrounding the defective pixels DP is computed and substituted for the pixel values of all of the defective pixels DP.
The method (b) above has the advantage of not relying on the operator's skill, but has the following drawback also.
In the event of only one defective pixel DP, this method is perfectly capable of correcting it without impairing texture. However, there are usually two or more, if any, defective pixels DP as shown in FIG. 2. The same pixel value substituted for the values of all of the defective pixels DP would result in a texture deterioration.
The defective pixels DP may exist in an area (called vignette) where the pixel values (tone) change smoothly. In such a case in particular, the same pixel value substituted for the values of all of the defective pixels DP would even create a large level difference to bring about a serious texture deterioration.